Onlooker Detection Users Guide
Table of Contents
Introduction
This example demonstrates the use of TI mmWave sensors for onlooker detection applications. Onlooker detection can be extremely useful in protecting a laptop user’s privacy. Traditionally, camera based methods are computationally intensive and power hungry. By introducing radar to the system, the camera based detection can be removed or can be enabled only when a possible onlooker is detected by the radar. Radar enables a lower power laptop onlooker solution enabling a longer battery life.
Requirements
Hardware Requirements
| Item | Details |
|---|---|
| EVM | xWRL6432BOOST |
| Computer | PC with Windows 10. If a laptop is used, please use the ‘High Performance’ power plan in Windows. |
| Micro USB Cable |
NOTE:
Both AWRL6432 and IWRL6432 are supported and can be used interchangeably. Please consult the respective datasheets for details on the differences between the devices.
Software Requirements
| Tool | Version | Download Link |
|---|---|---|
| TI mmWave SDK | 5.1.x.x | TI mmWave SDK 5.1.x.x and all the related tools are required to be installed as specified in the mmWave SDK release notes |
| Uniflash | 8.1.1+ | Uniflash tool is used for flashing TI mmWave Radar devices. Download offline tool or use the Cloud version |
| MATLAB Runtime | 9.11 | Visualizer uses MATLAB Runtime download here |
Getting familiar with the device
⚠️ Run Out of Box Demo
Before continuing with this lab, users should first run the out of box demo for the EVM. This will enable users to gain familiarity with the sensor’s capabilities as well as the various tools used across all labs in the Radar Toolbox.
Quickstart
Follow the instructions for Hardware Setup of Flashing Mode
Flash the binary listed below using UniFlash. Follow the instructions for using UniFlash
presence_motion_detect.release.appimage
The binary listed above can be found at:
<MMWAVE_SDK_INSTALL_DIR>\examples\mmw_demo\presence_motion_detect\prebuilt_binaries\Follow the instructions for Hardware Setup of of Functional Mode
Mount the EVM at the top of the laptop. This can be accomplished using a smartphone clamp mount as seen below.
Connect the EVM to the PC with a micro USB cable.
Run the GUI executable located at
<RTB_INSTALL>/tools/Visualizers/Low_Power_Visualizer/lowpower_demo_visualizer_6432.exeIn the configuration window, select the same port used when flashing for both control port and data port.
NOTE:
The port can be found in Device Manager. The port labelled XDS110 Class Application/User UART should be used.
For the configuration file, click Browse and navigate to the configuration file (.cfg) used for this lab. The config file can be found at:
<RTB_INSTALL>/source/ti/examples/Onlooker_Detection/chirp_configs/minor_motion_indoor_highbw_5m_onlookerDetection.cfgEnsure that Load Configuration is selected. Click Done.
The configuration file will be sent and the device will start chirping. Please allow a few seconds for the GUI to open the plots and begin plotting data.
Configuration Statistics
| Parameter | Value |
|---|---|
| Range Resolution | 0.1046 m |
| Maximum Range | 6.03 m |
| Velocity Resolution | 6.14 m/s |
| Maximum Velocity | 3.07 m/s |
| Average Power Consumption | 4-5 mW* |
*to estimate power based on configuration settings see the Sensing Estimator tool
Developer’s Guide
This example uses the Out of Box Demo firmware for XWRL6432. Refer to the Out of Box Demo User Guide for instructions on importing, building, and executing the demo with Code Composer Studio (CCS) as well as detailed information regarding the output format of the data sent out via UART.
Modifying the Configuration
The configuration file (.cfg) includes a set of commands which are used to specify the scene boundaries (i.e. area of interest) in relation to the sensor position and may need to be modified accordingly. These commands and their respective parameters are listed below.
- mpdBoundaryBox [
BoxIndex] [Xmin] [Xmax] [Ymin] [yMax] [Zmin] [Zmax]- Xmin/Xmax - Indicate start and end point of box along X-axis, no detection considered outside of this range
- Ymin/Ymax - Indicate start and end point of box along Y-axis, no detection considered outside of this range
- Zmin/Zmax - Indicate start and end point of box along Z-axis, no detection considered outside of this range
- sensorPosition [
Z] [AzimuthTilt] [ElevationTilt]- Z - height of sensor.
- AzimuthTilt - horizontal rotation of sensor with respect to the back wall, as represented by FarY.
- ElevationTilt - vertical rotation of the sensor with respect the the ground.
- minorStateCfg [
pointThre1] [pointThre2] [snrThre2] [pointHistThre1] [pointHistThre2] [snrHistThre2] [histBufferSize] [minor2emptyThre]- pointThre1 - Number of detected points (in a single frame) needed in a zone to enter the motion/presence state. If the number of points exceed this threshold, no need to check the SNR.
- pointThre2 - Number of detected points needed in a zone to enter the motion/presence state. If the number of points exceeds this threshold, the
snrThre2criteria is checked. - snrThre2 - Minimum total SNR (linear) of detected points (in a single frame) in a zone to enter the motion/presence state if the
pointThre2criteria is also satisfied. - pointHistThre1 - Number of detected points (in a frame history buffer) needed in a zone to enter the motion/presence state. If the number of points exceed this threshold, no need to check the SNR.
- pointHistThre2 - Number of detected points (in a frame history buffer) needed in a zone to enter the motion/presence state. If the number of points exceeds this threshold, the
snrHistThre2criteria is checked. - snrHistThre2 - Minimum total SNR (linear) of detected points (in a frame history buffer) in a zone to enter the motion/presence state if the
pointHistThre2criteria is also satisfied. - histBufferSize - Size of the frame history buffer size (in frames) used in
pointHistThre1,pointHistThre2, andsnrHistThre2parameters. - minor2emptyThre - A motion status is preserved if it recorded at least one motion detection in the last
minor2emptyThreframes.
Understanding the Output
A presence/motion decision is made for each of the zones defined in the configuration file. If motion is detected within one of the defined zones the zone will be highlighted in the GUI as shown below. The number of zones, zone sizes, sensor position, and parameters which define the presence/motion detection are all configurable in the configuration file (.cfg).
Need More Help?
- Search for your issue or post a new question on the mmWave E2E forums
- See the SDK for more documentation on various algorithms used in this demo. Start at
<mmWave_sdk_install_dir>/docs/mmwave_sdk_module_documentation.html